Visual Servoing-based Registration of Multimodal Images

International audienceThis paper deals with mutual information-based numerical and physical registration of white light images vs. fluorescence images for microrobotic laser microphonosurgery of the vocal folds. More precisely, it presents two techniques: a numerical registration of multimodal images and a vision feedback control for positioning an endoscope with regards to a preoperative image (fluorescence image). Nelder-Mead Simplex for nonlinear optimization is used to minimize the cost-function. The proposed methods are successfully validated in an experimental setup using preoperative fluorescence images and real-time white light images of the vocal folds

CAD model based tracking and 3D visual-based control for MEMS microassembly

This paper investigates sequential robotic microassembly for the construction of 3D micro-electro-mechanical systems (MEMS) structures using a 3D visual servoing approach. The previous solutions proposed in the literature for these kinds of problems are based on 2D visual control because of the lack of precise and robust 3D measures from the work scene. In this paper, the relevance of the real-time 3D visual tracking method and the 3D vision-based control law proposed is demonstrated. The 3D poses of the MEMS are supplied in real-time by a computer-aided design (CAD) model-based tracking algorithm. This latter is sufficiently accurate and robust to enable a precise regulation toward zero of the 3D error using the proposed pose-based visual servoing approach. Experiments on a microrobotic setup have been carried out to achieve assemblies of two or more 400 µm × 400 µm × 100 µm silicon micro-objects by their respective 97 µm × 97 µm × 100 µm notches with an assembly clearance from 1 µm to5µm. The different microassembly processes are performed with a mean error of 0.3 µm in position and 0.35×10 −2 rad in orientation

Online robust endomicroscopy video mosaicking using robot prior

International audienceThis paper discusses the development of a mosaicking algorithm for building large and high resolution confocal images. Due to the nature of optics and vision systems in general, there is still a dilemma between choosing a wide field-of-view (FOV) and high-resolution. The most accepted solution is to opt for a high-resolution optics and expand the FOV algorithmically thanks to mosaicking approaches. The study reported in this paper consists of online and real-time construction of large mosaics using individual confocal images with a micrometer resolution. These individual images are provided by a confocal laser endomicroscopy system which can grab in vivo real-time images through a minimally invasive access. The acquisition of the confocal images is achieved by moving the imaging probe on the studied sample surface with a constant contact between the probe and the sample. The mosaicking algorithm proposed in this paper deals with the combination of both the robot inputs and the image registrations. The proposed method has demonstrated very promising performances in terms of accuracy and robustness with regard to image noise (poor image quality or loss of contact between the probe and the sample) as well as misregistration issues. Experiments carried out with a highly accurate robotic system and a ground truth obtained by conventional optical microscopy demonstrate the robustness of the proposed approach

Handling and Manipulation of Microcomponents: Work-Cell Design and Preliminary Experiments

The paper introduces an experimental setup for the automatic manipulation of microcomponents, based on a 4 dof robot with Shoenflies motion and a two-camera vision system. The general architecture of the work-cell is presented. The work-cell functionality was tested via repeatability experiments using a set of vacuum grippers. Due to their intrinsic simplicity, vacuum grippers are very cheap and appear a promising solution for micromanipulation. An innovative nozzle for a vacuum gripper was designed, fabricated and tested, comparing its performance with traditional needles. The design was conceived to reduce the frequency of occlusions of the gripper and handle a wide range of particles. The performed tests evaluate the success and precision of the part release. Indeed, this is a crucial aspect of micromanipulation because microparts tend to stick to the gripper preventing the successful performance of manipulation tasks. The results confirm that adhesive effects prevent the release of components. For this reason different strategies were adopted in order to improve the efficiency in the release of stuck components. This solution increases the percentage of release and, setting appropriately the intensity of the pressure, it does not affect negatively the accuracy nor the repeatability of the positioning